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Several  important  and  efficient  methods  of  solution  of  specific  types 
of  linear  programming  problems,  such  as  the  "flooding"  o*  "flow"  methods  of 

Boldyrc-ff  and  Ford-Fulkerson  for  network  flows  and  tne  "decomposition" 
principle  or  "mixing  routines"  of  Dantzig-'-Volfc  or  Charnes-Cooper,  have  the 
distressing  feature  for  managerial  applications  of  sometimes  providing 
optimal  solutions  which  are  not  extreme  point  (or  basic)  solutions.  Thereby, 
any  immediate  and  automatic  availability  of  the  optimal  dual  evaluators  of  the 
original  problem  is  not  forthcoming.  Indeed,  several  such  experiences  have 
resulted  in  abandonment  of  these  techniques  for  managerial  studies  in  certain 
industrial  firms. 

Aside  from  sensitivity  analysis,  it  is  also  often  desirable  to  start  from 
an  operable  ("feasible")  solution,  not  necessarily  optimal,  which  is  either 
suggested  by  company  personnel  or  from  various  pertinent  qualitative  consid¬ 
erations,  and  to  be  able  to  proceed  automatically  to  an  extreme  point 
(=  basic j  solution  which  is  at  least  as  good  as  the  suggested  one.  ^ 

( Thereafter  any  one  of  the  various  methods  could  be  employed  to  achieve 
optimal ity. ) 

The  purpose  of  this  paper  is  to  exhibit  how  part  of  the  technique  of  proof 

2/ 

of  the  opposite  sign  theorem  -  can  be  employed  in  a  simple  algorithmic 
manner  to  achieve  this  end.  'Ve  present  an  ALGOL  code  for  executing  this 
algorithm  in  a  manner  compatible  (as  a  procedure)  with  standard  programs. 

Consider  for  the  moment  the  linear  programming  problem  written  in  the 
form : 


1/  There  have  been  constructive  methods  which  reduce  any  feasible  solution 
to  a  basic  solution,  but  which  have  not  considered  this  important 
feature,  or  other  features  such  as  ability  to  employ  a  known  but  not 
necessarily  feasible  basis. 

2/  See  reference  [3]  on  semi-infinite  programming  for  a  proof  of  it  in 
general  form. 


maxf(\}=Z  c.\. 

j=l  ^  J 


'subject  to  Z  P  \ P 


\  >  0 


where  P .  ,  P  are  al  1  m-vcctor  ..  Let  Z  PA.-('  >0]. 

JO  J  j  0  j  -  ^ 

-Ve  may  assume  tnat  the  problem  is  regularized.  This  means  that  the 
solution  set  is  non-empty  and  bounded,  so  the  Opposite  Sign  Theorem  applies: 


A  is  generated  by  its  extreme  points  : 


I  Z  a.P.=  0,notall 
j=l  ^  ^ 


a  .  =  0 
J 


>  some  are  of  opposite  sign. 


’■Jow  suppose  some  collection  B=  ,  q  <  m]  is  a  basis 

for  the  sutspace  of  Fi^  spanned  by  tlie  P^.  (Some  or  all  of  the  v.  and  P^  may 
he  identical;  it  is  not  required  that  B  be  composed  oniv  of  certain  PAs./ 
For  notational  convenience,  rename  any  which  is  not  present  in  the 
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collection  jrl,...,n|  as  and  let  1=  ^i:P^  e  b]  , 

Let  S'  denote  a  left  inverse  of  the  m  >  q  basis  matrix  B*  then 

T 

^  1^  is  the  expression  of  relative  to  B,  or 

T,  =  ’’k  • 

It  I 


so  that  Z  a .  P.  a,  P,  =  0, 
•,  T  11  k  k 
le  I 


where  a,  =  -1. 
k 


Now  P'^  can  replace  ary  r,  in  the  basis  with  a.  ^  0,  and  by  tne  ofposit^' 

sigri  theorom  some  and  are  of  opposite  sign. 

I  et  \  be  any  solution  to  lA-i  ,  Assume  that  some  vector  P.  ^  B 

0  •  ^ 

occurs  with  positive  \-comf)cnent  in  the  expression  of  P^;  if  no  such 
vector  exists  then  w-e  already  tiave  an  extreme  point  solution. 

.<p  now  m.ust  considt^r  th.e  two  possibilities  which  arise  relative  to  th** 
values  of  the*  corres'poncH ng  and  components: 


1/  -jCo  Charnos-Cfxspf'r ,  reference  12],  p.  424. 

2/  oee  L  4] ;  note  *that  tlie  matrix  B  need  not  be  square. 
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Casel. 


There  exists  some  ^  0  with  corresponding  \^  =  0. 


i 


Choose  one  of  these,  say  ,  to  designate  a  "remove"  vector.  No  change 
will  be  effected  in  the  functional  to  be  considered  until  we  have  only  P.'s 
in  the  basis;  until  this  occurs  we  need  not  be  concerned  with  the  opposite 
sign  property  but  are  only  choosing  a  basis  from  among  the  vectors. 

Case  2.  For  every  a.  ^ 


,  0  . 

X  . 

min  ^  and 

p  =  min  , 

X-.>0  a. 

aJ>0  J 

X.>0  la.  1 
aj<0  1  J  I 

whose  existence  is  guaranteed  by  the  opposite  sign  theorem.  Then 
1  2 

X.  =  \  -  p^a  ,  X  =  X+  p^a  will  have  every  component  >  0  and  each  will  have 
at  least  one  more  zero  component  than  X  as  long  as  the  number  of  \^>0  is 
greater  than  q.  Note  also  that  because  X  is  a  solution  it  is  true  that 

X  +  pa  is  also  a  solution  for  any  real  p  ,  since  ^  a.  P.  =  0  ,  so  that  in 

12  i  ^ 

particular  X  and  X  are  both  feasible  solutions. 

1  2 

Now  let  f(X)  be  convex  ,  i.e.  ,  for  any  X  ,  X 

f((l-Y)'^^  +  y’^^)  <  (l^)f(X^)  +  ,  0  <  Y  <  1  holds. 

^2  1  ^12 

Since  X  =  _  X  +  _  X  ,  it  follows  that 

Pl^P2  Pi^P2 

Po  1  Pi  2 

f(\)  <  _  f(X^)  _  f(X^;  ,  so  that  not  both 

Pl^P2  P1^P2 

f(X^)  <  f(X)  and  f(X^/  <  f(XJ  ;  i.e.,  one  of  X^  ,  X^  yields  at  least  as 
great  a  functional  value  as  X.  Thus,  the  j  which  yields  the  minimum  in  the 
definition  of  p^  ,  when  f(X^)  >  (or  of  when  f(X^)  >  f(X^)) 

serves  to  designate  a  vector  P^  which  is  to  be  replaced  by  Pj^  in  the  basis. 

At  this  point  we  begin  again  with  the  new  basis  and  new  feasible  X  as  before. 

Thus  at  each  stage  we  remove  one  or  more  vectors  from  the  original  set, 
while  maintaining  at  least  as  great  a  value  as  in  the  previous  stage,  until 
the  original  set  is  reduced  to  a  set  of  linearly  independent  vectors,  i.e., 
the  corresponding  X  is  an  extreme  point  with  at  least  as  great  a  functional 
value  as  the  original  given  solution.  (A  corresponding  result  holds  when  a 
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concave  functional  is  be  be  minimized.)  We  can  summarize  the  above  discussion 
in  the  following  algorithm: 

n 

Step  1.  Start  with  a  feasible  solution  \  ,  Z  PA  .  =  Pg  »  ^  ^  0  , 

j=l  J  J 

and  a  basis  B=  ,  ie  l]  . 

Step  2.  Find  any  ^|^>0  with  to  designate  a  vector  to  enter  the 

basis;  if  none  is  available,  \  is  an  extreme  point  and  the 
process  terminates. 


Step  3.  Obtain  the  in  Z  Oj^P^  +  Oj^Pj^  =  0  ,  where  Oj^  =  -1  » 

T  h 

by  computing  a  =  B  P^^  . 

Step  4.  (i)  If  there  exist  /  0  with  corresponding  =  0  ,  choose 
one,  say  »  to  designate  a  remove  vector  P^  .  Replace  P^ 
by  P|^  in  B,  compute  the  new  B^  ,  and  go  to  step  2. 


(ii)  Otherwise  set  = 

a 
\ 

Step  5.  Let  =  X-p^a  and  X^^^  =  X+p^o  •  Let  ,  r^  be  a  pair 

of  indices  for  which  p^  ,  p^  achieve  their  respective  minima. 
Choose  P  or  P  to  be  removed  (  =  t  )  according  as 

f(X^^^)  or  f(\  is  the  larger. 

Step  6.  Substitute  P,  for  P  in  the  basis,  compute  the  new  B^  ,  and 

^  ^  (i) 

return  to  step  1  with  X  '  as  a  new  X  . 

Observe  that  the  algorithm  terminates  when  a  X  is  reached  such  that  each 

P,  with  X,  >0  is  in  the  current  basis, 
k  k 

Notice  again  that  it  is  possible  to  choose  an  initial  basis  consisting 
entirely  of  artificial  or  slack  vectors  (even  though  they  may  not  be  part  of 
the  original  set  of  vectors),  and  that  the  procedure  will  automatically  find 
a  basic  solution  in  terms  of  the  original  vectors. 

It  should  be  expressly  noted  that  the  process  of  choosing  a  vector  to 
enter  the  basis  (any  vector  with  X^X)  ,  and  not  already  in  the  basis,  may  be 
chosen),  computing  a  column  of  Qj  ("substitution  ratios"),  selecting  a 


vector  to  be  removed  from  the  basis  (the  one  at  which  either  the  or  the 
minimum  ratio  was  achieved),  and  transforming  the  matrix  of  the  basis  inverse 
to  reflect  thxs  change  of  basis  corresponds  closely  in  structure  with  that  of 
the  modified  simplex  method  of  A.  Charnes  and  C.  E.  Lemke,  although  the 
actual  criteria  governing  these  operations  are  different.  In  particular,  just 
as  in  the  modified  simplex  method,  the  possibility  of  an  unbounded  solution 
is  indicated  by  the  absence  of  a  positive  (pivot  element),  although, 
since  there  are  normally  two  candidates  for  removal  from  the  basis,  this 
condition  is  critical  only  if  the  other  candidate  would  cause  a  worsening  of 
the  solution;  e.g.  ,  if  we  are  maximizing,  the  availability  of  an  infinite 
minimum  will  not  cause  an  error  stop;  we  would  simply  remove  the  vector 
associated  with  the  minimum  ratio  from  the  basis  if  no  decrease  in  the 
functional  is  caused  thereby. 

It  is  conceivable  that  the  initial  "solution"  presented  to  the 
procedure  could  be  really  not  a  solution  at  all.  If  K  is  the  "solution" 
presented  and  P\  =  P  ^  ,  the  resulting  "basic  solution"  will  be  a  basic 

solution  to  P\  =  P  ,  since  the  algorithm  never  refers  to  the  actual 
stipulations  vector  P^  .'  For  production  use,  therefore,  it  would  be 
;vorth  while  to  check  before  using  the  procedure  that  the  solution  proposed 
really  is  feasible. 

The  ALGOL  procedure  presented  below  assumes  that  f(\)  is  linear, 
i.e.  ,  f(X.)  =  c\  ,  which  is  both  concave  and  convex.  The  roles  of  the 
formal  parameters  used  in  the  procedure  declaration  are  described  in  the 
following  tables 
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Formal  1 arameter 

I  Description 

m 

The  number  of  constraints.  (The  number  of 

rows  In  P,  J 


n 

The  number  of  columns  in  P,  exclusive  of  P 

0 

size 

The  number  of  vectors  in  the  basis,  which  is 
the  dimension  of  the  space  spanned  by  the 

Pj  .  In  most  cases  this  will  be  equal  to  m 

basis 

The  list  of  indices  of  the  P.  in  the 

starting  basis;  if  no  starting  basis  is 
supplied  then  the  elements  of  this  array 
are  irrelevant  when  the  procedure  is  called 

matrix 

1 

The  m  X  n  constraint  matrix. 

inverse 

A  size  x  m  (left)  inverse  of  the  initial 
basis;  again,  the  elements  are  irrelevant 
if  no  basis  is  specified.  One  additional 
row  should  be  adjoined  wnich  will  contdin 
the  dual  evaluators. 

lambda 

The  initial  feasible  solution,  whose 
components  must  be  in  the  same  order  as 
the  columns  of  the  coefficient  matrix. 

c 

The  components  of  the  objective,  in  the 
same  order  as  those  of  lambda. 

1 

objective 

An  integer  which  should  be  zero  or  one  if 
minimization  or  mjximization ,  respectively, 
is  desired. 

option 

An  indicator  which  should  be  set  to  one  if 
no  starting  basis  is  specified  in  which 
ca«i?  the  m  X  m  identity  matrix  will  be  userl 
so  that  size  must  be  eou^l  to  m;  if  a 
basis  and  inverse  are  provided,  tliis 
indicator  should  be  set  *■->  2. 

unbounded 


The  name  of  a  sentinel  which  will  normally 
be  set  to  zero  but  wlich  assumes  a  non¬ 
zero  value  in  case  an  infinite  optimum  is 
attainable. 
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In  terms  of  the  ALGOL  identifiers  used  as  formal  parameters  of  the 
procedure,  upon  completion  of  the  process  the  integer  <jrray,  basis  , 
contains  the  numbers  of  the  vectors  in  the  basic  solution*  the  array, 
lambda  ,  contains  the  actual  values  of  the  .  ,  and  unbounded  is 
equal  to  D;  if  not  zero,  it  was  set  to  the  number  of  the  vector  attempting 
to  enter  the  basis  when  unboundedness  was  noticed  and  the  process  terminated. 
The  basis  (left;  inverse  is  contained  in  inverse  ,  which  also  includes  a 
size  +  1^^  row  containing  the  dual  evaluators  for  the  current  basic 
solution. 

ALGOL  F rocedure 

procedure  purify  (m,  n,  size,  basis,  matrix,  inverse,  lambda,  c,  objective, 

option,  unbounded;  j 

value  m,  n,  size,  objective,  option  ; 

integer  m,  n,  size,  objective,  option,  unbounded  • 

real  array  matrix,  inverse,  lambda,  c  ; 

integer  array  basis  ; 

begin  integer  i,  j,  k,  r,  rl ,  t7  ; 

array  alpha  [1  ;  size  +  l]  • 

real  rhol ,  rho2,  t,  dl,  d2  ; 

unbounded  »  =  0  ; 

comment  if  necessary,  generate  augmented  identity  matrix  for  the 
left  inverse  and  dual  evaluators  • 

if  option  =  1  then  for  j  ;  =  1  step  1  until  n  do 

begin  for  i  i  =  1  step  1  until  m  +  1  ^ 

inverse  L  i  ,  j  ]  ;  =  if.  i  -  J  then  1  else  0  ; 

basis  [jj  ;  =  rn-j 

end 

look  for  some  lambda  greater  than  zero  to  determine  a  vector 
to  enter  the  basis  • 
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ayain  ;  for  j  :  =1  1  until  n  ^ 

lambda  [j]  then  lambda  [j]  :  =  o 

else  begin  for  i  x  =1  step  1  until  size  do 


if  basis  [i]  =j  then  goto  skip  it  ; 

comment  if  we  get  to  here,  vector  j  was 
not  already  in  the  basis  ; 

k;  =j  ; 


loto  come  in  ; 


skip  it;  end 

if  we  get  to  here,  no  vector  k  can  be  found  with  lambda  [k]  >  0  and 
which  is  not  already  in  the  basis,  hence  we  are  done,  ; 

goto  finished  ; 

come  in:  for  ir:l  step  1  until  size  +  1  do 


begin  alpha  [i]  ;  =  0  ; 


for  j  :  =  1  step  1  until  m  ^ 

alpha  Lil  :  =  alpha  [i]  +  inverse  Li,j]*  matrix  [j,k] 


£IL1  ; 


alpha  [size  +  l]  ;  =  alpha  [size  +  l]  -  c[k]  ; 


rhol  ;  =  rh02  :  =  20  } 

comme nt  now  find  two  candidates  for  removal  ; 

for  i  :  =  1  step  1  until  size  ^ 

H  alpha  [i]  >  -7  then 

begin  if  basis  [i]  >  n  then 

begin  r  ;  =  i  ;  goto  transform  end 

el se  if  lambda  [basis  [i]  J  ^  then 

begin  r  ;  r  i  j  goto  transform  end 


* 


Pise  begin  t  ;  =  lambd.i  [basis  [il  J  /  alf  ha  [ii  • 
^  t  <  rhol  then 
begin  rhol  :  r  t  j  rl  ;  -  i  .tH 

eni 

end 

else  i f  alpha  [i]  <  -7  then 

begin  i f  oasis  [i]  >  n  then 

begin  r  :  =  i  ;  goto  transform  end 

else  i f  lambda  [basis  [i]  J  <  ^  ^  -/  tfi'ni 

begin  r  j  =  i  ;  goto  traisfori'i  end 
else  begin  t  :  =  -  lambda  [basis  [ij  J  /  alv.ha  [ij  • 

t  <  rho2  then 

beg i n  rho2  :  =  t  ;  r2  :  =  i  ► n i 
end 

end  ; 

i f  lame  da  [k]  <  rho2  then 

beg i n  rho2  lambda  [k]  •  r2  :  =  -1  en  i  ; 

dl  ;  =  c[kj  *  rhol  ; 
d2  ;  =  -c[k]  *  rho2  ; 
for  i  :  =  1  step  1  until  size  do 
begin  t  c  [basis  [i]  ]  •  alpha  [i]  ; 
dl  ;  =  dl  -  rhol  *  t  ; 
d2  ;  =  dr  f  rho2  *  t 


^  ; 

comme nt  now  cfioosc  tfie  best  vector  to  be  renoved,  eii.'jfr  rl  or  r?  ; 
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if  objective  =  0  =  dl  <  d2 
then  goto  takeout  1  else  goto  takeout  2  j 
takeout  1:  i_f  rhol  =  20  then  begin  unbounded  :  =  k  ;  goto  finished  end  ; 

for  i  j  =  1  step  1  until  size  ^ 

lambda  [basis  [i]  ]  :  =  lambda  [basis  [i]  ]  -  rhol  *  alpha  [i]; 
lambda  [k]  :  =  lambda  [k]  +  rhol  ; 
r  :  =  r  1  ; 
goto  transform  ; 

takeout  2;  for  i  ;  =  1  step  1  until  size  ^ 

lambda  Lbasis  [i]  ]  :  =  lambda  (.basis  [i]  ]  +  rho2  *  alpha  [i]; 

lambda  [k]  :  =  lambda  [k]  -  rho2  • 
if.  r2  =  -1  then  goto  again  else  r  :  =  r2  ; 
transform:  for  j  :  =  1  step  1  until  m  do 

begin  inverse  |.r,j]  :  =  inverse  [r,j]  /  alpha  [r]  ; 

I 

for  i  ;  =  1  step  1  until  size  +  1  do 

I 

ii  i  ^  r  then 

inverse  L^tj]  -  inverse  [i,j]  -  inverse  [r,j]  *  alpha  [^ 

end  • 

basis  [r]  ;  =  k  ; 

goto  again  ; 
finished;  end  j 

Example 

.<e  present  a  simple  example  with  a  summary  of  the  iterations  taken  to 
reach  a  basic  solution,  starting  from  an  initial  basis  not  consisting  of 
vectors  taken  from  the  original  coefficient  matrix. 
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max  2x,  +  x_  -t-  4x„  +  3x  +  x_ 

12345 

subject  to  5x  +  x^  -t-  6x.  +  3x  =  26 

^  4.  3 

7x,  +  X-  +  2x-  -  X.  -  2x-  =  5 
1  2  3  4  5 

and  all  x.  >  0 
1  - 


Come-in  Remove  CXirrent 


Staoe 

Vector 

Vector 

Basis 

\ 

f(\) 

Case 

Initial 

— 

— 

(i  (A 
lo  1 

(2,1, 1,6, 3, 0,0) 

30 

— 

1 

^6 

(2, 1,1,6, 3, 0,0) 

30 

Case  1 

2 

‘"2 

/5  1^ 

V  1/ 

(2, 1,1, 6, 3, 0,0) 

30 

Case  1 

3 

^^3 

,1  6^ 

',1  2) 

(0,17,0,6,3,0,0) 

38 

Case  2 

4 

P4 

^3 

S-?) 

(0,17,0,6,3,0,0) 

38 

Case  1 

5 

'5 

li-v 

(0,26,0,21,0,0,0) 

89 

Case  2 

The  opt imum  value  for  this  problem  is  104  -  3/5 
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